Image Processing Using Opencv

---

Install Opencv

• Example on MacOS

$ pip3 --version
pip 24.0 from /opt/anaconda3/lib/python3.12/site-packages/pip (python 3.12)
$ python3 --version
Python 3.12.4
$ pip3 install opencv-python
Collecting opencv-python
  Downloading opencv-python-4.11.0.86.tar.gz (95.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 95.2/95.2 MB 2.7 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy>=1.21.2 in /opt/anaconda3/lib/python3.12/site-packages (from opencv-python) (1.26.4)
Building wheels for collected packages: opencv-python
  Building wheel for opencv-python (pyproject.toml) ... done
  Created wheel for opencv-python: filename=opencv_python-4.11.0.86-cp312-cp312-macosx_10_16_x86_64.whl size=27595148 sha256=0fc6f3f12d5b5f0a5a54d870de9e866aaf59484b11dfcce77a8aece5a4f393e9
  Stored in directory: /Users/bshen2/Library/Caches/pip/wheels/be/bd/d5/425eca52f204ab4b1ad7ac23c79e7a0458ee178056e4350265
Successfully built opencv-python
Installing collected packages: opencv-python
Successfully installed opencv-python-4.11.0.86

---

Filtering through Convolution

Let's use convolution for the following image processing:

• Edge Detection
• Feature Extraction
• Blurring → Scaling

Import resources and display image

import matplotlib.pyplot as plt
import matplotlib.image as mpimg

import cv2
import numpy as np

%matplotlib inline

# Read in the image
image = mpimg.imread('building2.jpg')
#image = mpimg.imread('test.png')

plt.imshow(image)

<matplotlib.image.AxesImage at 0x20304041c10>

Convert the image to grayscale

# Convert to grayscale for filtering
gray = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)

plt.imshow(gray, cmap='gray')

<matplotlib.image.AxesImage at 0x20309fad5b0>

Convolution Through a Kernel

Image convolution is a mathematical operation where a small matrix (called a kernel or filter) slides over an image, performing element-wise multiplications and summing the results to produce a new pixel value. This process helps extract features such as edges, textures, and patterns by emphasizing specific spatial structures in the image. Convolution is widely used in image processing and deep learning, particularly in convolutional neural networks (CNNs) for feature detection.

Here we practice applying image convolution using traditional image processing operations.

1. Select a Kernel (Filter): Choose a small matrix (e.g., 3x3 or 5x5) with predefined values for a specific operation (e.g., edge detection, blurring).

2. Slide the Kernel Over the Image: Move the kernel across the image, covering one region at a time.

3. Compute the Element-Wise Product: Multiply each value in the kernel with the corresponding pixel values in the image region.

4. Sum the Products: Add up all the multiplied values to obtain a single new pixel value.

5. Store the Result: Place the computed value into the corresponding location in the output image.

6. Repeat for Entire Image: Continue the process by shifting the kernel until every pixel has been processed.

Here is a common 3×3 kernel for edge detection, which detects edges in a specific direction, vertical and horizontal:

Vertical Edge Detection Kernel:

$

K_x = \begin{bmatrix} -1 & 0 & 1 \\ -1 & 0 & 1 \\ -1 & 0 & 1 \end{bmatrix} $

Horizontal Edge Detection Kernel:

$

K_y = \begin{bmatrix} -1 & -1 & -1 \\ 0 & 0 & 0 \\ 1 & 1 & 1 \end{bmatrix} $

These kernels are convolved with an image to highlight vertical and horizontal edges, respectively.

Applying a filter like this to an image is a way of taking (an approximation) of the derivative of the image in the x or y direction, separately.

# Create a custom kernel

# 3x3 array for edge detection
horizon = np.array([[ -1, -1, -1], 
                    [  0,  0,  0], 
                    [  1,  1,  1]])

## TODO: Create and apply a vertical edge detection operator
vertical = np.array([[ -1, 0, 1], 
                    [ -1, 0, 1], 
                    [ -1, 0, 1]])

diag_45 = np.array([[ -2, -1, 0], 
                    [ -1,  0, 1], 
                    [  0,  1, 2]])

diag135 = np.array([[ 0, -1, -2], 
                    [ 1,  0, -1], 
                    [ 2,  1,  0]])

fig = plt.figure(figsize=(24,24))
# Filter the image using filter2D, which has inputs: (grayscale image, bit-depth, kernel)  
filtered_image = cv2.filter2D(gray, -1, horizon)
fig.add_subplot(2,2,1)
plt.imshow(filtered_image, cmap='gray')
plt.title('horizontal')

filtered_image2 = cv2.filter2D(gray, -1, vertical)
fig.add_subplot(2,2,2)
plt.imshow(filtered_image2, cmap='gray')
plt.title('vertical')

filtered_image3 = cv2.filter2D(gray, -1, diag_45)
fig.add_subplot(1,2,1)
plt.imshow(filtered_image3, cmap='gray')
plt.title('diag_45')

filtered_image4 = cv2.filter2D(gray, -1, diag135)
fig.add_subplot(1,2,2)
plt.imshow(filtered_image4, cmap='gray')
plt.title('diag135')

plt.show()

# Create a custom kernel for blurring

# 2x2 kernel for averaging blurring 
S2x2 = np.array([[ 1, 1], 
                 [ 1, 1]])

S3x3 = np.ones((3, 3))/9 #just based off of the 2x2 example
S5x5 = np.ones((5, 5))/25

fig = plt.figure(figsize=(48, 12))
fig.add_subplot(4,1,1)
plt.imshow(gray, cmap='gray')
plt.title('original')

# Filter the image using filter2D, which has inputs: (grayscale image, bit-depth, kernel)  
blurred_image = cv2.filter2D(gray, -1, S2x2/4.0)
fig.add_subplot(4,1,2)
plt.imshow(blurred_image, cmap='gray')
plt.title('2x2')

# TODO: blur image using a 3x3 average
fig.add_subplot(4,1,3)
blurred_image_3 = cv2.filter2D(gray, -1, S3x3)
plt.imshow(blurred_image_3, cmap='gray')
plt.title('3x3')

# TODO: blur image using a 5x5 average
fig.add_subplot(4,1,4)
blurred_image_5 = cv2.filter2D(gray, -1, S5x5)
plt.imshow(blurred_image_5, cmap='gray')
plt.title('5x5')

plt.tight_layout() #so it looks nicer
plt.show()

TODO

Other image processing/filtering you can try:

• Other Edge Detector (e.g. Sobel Operator)

A common 3×3 kernel for edge detection is the Sobel operator, which detects edges in a specific direction. Below are the Sobel kernels for detecting vertical and horizontal edges:

Vertical Edge Detection Kernel:

$

K_x = \begin{bmatrix} -1 & 0 & 1 \\ -2 & 0 & 2 \\ -1 & 0 & 1 \end{bmatrix} $

Horizontal Edge Detection Kernel:

$

K_y = \begin{bmatrix} -1 & -2 & -1 \\ 0 & 0 & 0 \\ 1 & 2 & 1 \end{bmatrix} $

These kernels are convolved with an image to highlight vertical and horizontal edges, respectively.

• Corner Detection (use the kernels we discussed in slides)
• Scaling (after the blurring, can you pick one pixel out of the following?)
  • 2x2
  • 4x4
• Use other images of your choice
• For a challenge, see if you can put the image through a series of filters: first one that blurs the image (takes an average of pixels), and then one that detects the edges.

#sobel edge detection kernels

# used earlier edge detection to do cells and replaced kernels with sobel kernels
sobel_horizon = np.array([[ -1, -2, -1], 
                    [  0,  0,  0], 
                    [  1,  2,  1]])

sobel_vertical = np.array([[ -1, 0, 1], 
                        [ -2, 0, 2], 
                        [ -1, 0, 1]])


fig = plt.figure(figsize=(24,24))

filtered_image = cv2.filter2D(gray, -1, sobel_horizon)
fig.add_subplot(2,2,1)
plt.imshow(filtered_image, cmap='gray')
plt.title('sobel horizontal')

filtered_image2 = cv2.filter2D(gray, -1, sobel_vertical)
fig.add_subplot(2,2,2)
plt.imshow(filtered_image2, cmap='gray')
plt.title('sobel vertical')


plt.show()

#corner detection

kernels = [
    np.array([[0, 1, 0],
              [1, 1, 0],
              [0, 0, 0]]),

    np.array([[0, 1, 0],
              [0, 1, 1],
              [0, 0, 0]]),

    np.array([[0, 0, 0],
              [0, 1, 0],
              [1, 1, 0]]),

    np.array([[0, 0, 0],
              [0, 1, 0],
              [0, 1, 1]])
]

corner_response = np.zeros_like(gray)
for k in kernels:
    response = convolve2d(gray, k, mode='same', boundary='symm')
    corner_response = np.maximum(corner_response, response)

corner_response = (corner_response - corner_response.min()) / (corner_response.ptp()) 
threshold = 0.9
y, x = np.where(corner_response > threshold)

plt.figure(figsize=(10, 6))
plt.imshow(gray, cmap='gray')
plt.scatter(x, y, c='red', s=6)
plt.title("Corner Detection")
plt.show()

#scaling
def downscale(img, factor):
    return img[::factor, ::factor]

downscaled_2 = downscale(gray, 2)
downscaled_4 = downscale(gray, 4)

plt.figure(figsize=(48, 48))
plt.subplot(4, 3, 1)
plt.imshow(gray, cmap='gray')
plt.title("Original")

plt.subplot(4, 3, 2)
plt.imshow(downscaled_2, cmap='gray')
plt.title("Downscale 2x2")

plt.subplot(4, 3, 3)
plt.imshow(downscaled_4, cmap='gray')
plt.title("Downscale 4x4")

#plt.tight_layout()
plt.show()

#series of filters

S3x3 = np.ones((3, 3))/9 #just based off of the 2x2 example

fig = plt.figure(figsize=(48, 12))
fig.add_subplot(4,1,1)
plt.imshow(gray, cmap='gray')
plt.title('original')

#blur image using a 3x3 average
fig.add_subplot(4,1,2)
blurred_image_3 = cv2.filter2D(gray, -1, S3x3)
plt.imshow(blurred_image_3, cmap='gray')
plt.title('3x3 blur')

#sobel edge detection kernels

# used earlier edge detection to do cells and replaced kernels with sobel kernels
sobel_horizon = np.array([[ -1, -2, -1], 
                    [  0,  0,  0], 
                    [  1,  2,  1]])

sobel_vertical = np.array([[ -1, 0, 1], 
                        [ -2, 0, 2], 
                        [ -1, 0, 1]])


filtered_image = cv2.filter2D(blurred_image_3, -1, sobel_horizon)
fig.add_subplot(4,1,3)
plt.imshow(filtered_image, cmap='gray')
plt.title('3x3 blur + sobel horizontal')

filtered_image2 = cv2.filter2D(blurred_image_3, -1, sobel_vertical)
fig.add_subplot(4,1,4)
plt.imshow(filtered_image2, cmap='gray')
plt.title('3x3 blur + sobel vertical')


plt.tight_layout() #so it looks nicer
plt.show()